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ABSTRACT 


Considerable  research  is  currently  going  on  into 
the  application  of  distributed  computing  systems. 
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needs  of  a  small  sarship.  The  particular  constraints 
of  the  warship's  environment  are  discussed.  This  is 
followed  ty  a  description  of  how  a  ring  structured 
distributed  computing  system  might  be  adapted  tc 
function  in  this  environment.  Included  in  this 
consideration  are  the  feasibility  of  attaining 
adequate  bus  speed,  the  use  of  multiply  addressed 
messages,  and  methods  of  handling  real-time 
processing.  Of  particular  interest  is  the  ability  to 
achieve  controlled  degradation  of  performance  under 
failure,  especially  failure  due  to  battle  damage. 
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I.   INTRODUCTION 


With  the  advance  of  large  scale  integration  technology 
and  the  resulting  decreasing  ccst  of  digital  processing 
elements,  the  concept  of  using  a  number  of  small  processors 
tied  together  by  seme  form  of  data  bus,  rather  than  using 
one  large  computer,  has  been  receiving  increasing  interest. 
The  advantages  of  this  type  of  system  include  greater 
flexibility  in  the  system,  lower  costs,  and  increased 
reliability.  Eistribiated  computing  systems  may  gain  some  or 
all  of  these  advantages  depending  on  the  hardware  and 
software  utilized  to  bind  the  processors  together  intc  the 
system. 

The  advantages  mentioned  would  be  particularly  useful 
for  application  tc  a  warship's  computing  requirements. 
Perhaps  the  foremost  advantage  in  a  world  of  rapidly 
changing  technology  that  makes  warships  virtually 
obsolescent  before  tiey  can  be  made  operational,  is  the  ease 
with  which  a  properJLy  designed  system  could  be  reconfigured. 
When  technigues  for  dynamically  reconfiguring  the  system  in 
the  event  cf  normal  failure  or  battle  damage  are  considered 
as  well,  the  concept  of  a  distributed  computing  system 
appear  particularly  attractive. 

Farber  £5,6]  has  been  investigating  ring  structured 
distributed  computing  systems  for  several  years.  In 
reference  5  he  discusses  the  concept  of  "fail  soft"  systems, 
that  is,  systems  which  exhibit  the  property  of  controlled 
system  degradation,  rather  than  catastrophic  failure,  as  a 
result  cf  component  failure.  In  addition,  a  prototype  ring 
utilizing    many   of   these   principles   and   employing 


iiicroprocesser  technology  has  been  designed  at  the  Naval 
Postgraduate  School  .(NPS)  [1]-  It  is  not  the  intent  here  to 
reiterate  arguments  regarding  the  overall  advantages  of  a 
ring  structured  system  with  respect  to  reliability  and 
flexibility.  These  are  covered  in  the  references.  What  is 
intended  is  to  investigate  the  modifications  and  extensions 
tc  such  a  system  which  would  be  required  to  adapt  it  tc  the 
computing  Deeds  cf  a  small  warship. 


A.   EASIC  CONCEPTS  OF  A  RING  STRUCTURED  DISTRIBUTED 
CCCEUTING  SYSTEM 


The  heart  of  the  distributed  computing  system  is  the 
method  of  communication  between  the  processors.  To  gain  the 
maximum  in  flexibility  and  fault  tolerance  the  system 
should: 

1.  avcid  centralization  cf  control, 

2.  permit  communication  between  processes  without  regard 
for  physical  location  of  the  process, 

3.  permit  execution  of  processes  without  regard  for 
their  physical  location. 

The  last  two  ease  the  job  of  dynamic  reconfiguration  of 
the  system  while  the  first  is  obviously  necessary  to  prevent 
the  failure  cf  a  particular  component  from  bringing  the 
whcle  systen  to  a  halt  (catastrophic  failure) .  To  achieve 
these  twc  goals  the  following  basic  system  is  proposed. 

The  ccmmunicatipn  network  consists  of  a  unidirectional 
ring  to  which  a  processor  is  attached  via  a  ring  interface. 
Only   the   ring  interface  in  control  can  transmit  a  message. 


On  completion,  control  is  passed  to  the  next  ring  interface. 
Thus,  ccntrcl  is  decentralized.  Independent  timers  in  the 
interfaces  ensure  that  no  one  ring  interface  monopolizes  the 
ring  and  that  if  ccntrol  is  lost,  it  will  be  regained  ry  one 
of  the  other  ring  interfaces. 

The  indication  tiat  a  ring  interface  may  take  control  is 
the  receipt  cf  a  special  message  called  a  control  token. 
Mhen  an  interface  has  no  message  to  transmit,  it  simply 
passes  the  token  on  to  the  next  ring  interface.  If  it  has  a 
message  to  send,  the  control  token  is  net  retransmitted. 
Instead,  the  message  is  placed  onto  the  ring,  is  passed 
completely  arcund  the  ring  and  removed  by  the  originating 
xiog  interface.  The  control  token  is  put  back  onto  the  ring 
after  the  end  of  the  message. 

Two  tckens,  the  start  of  message  (SCH)  and  end  of 
message  (ICE) ,  are  used  to  delineate  the  data  being  sent. 
The  SCM  is  followed  by  the  name  of  the  addressee.  The  data 
cones  next  and  then  the  EOM  token  followed  by  several  bits 
used  to  check  for  the  proper  receipt  of  the  message. 

Each  ring  interface  has  a  list  cf  the  processes  which 
are  operating  in  its  host  processor.  Messages  are  addressed 
to  these  processes*.  If  the  ring  interface  finds  a  match 
between  the  address  en  an  incoming  message  and  one  cf  the 
processes  in  its  list,  it  passes  the  message  to  its  host 
processor  as  well  as  passing  it  on  around  the  ring.  Status 
bits  appended  to  the  end  of  the  message  as  it  is  passed  on 
indicate  to  the  originating  ring  interface  whether  or  not  a 
message  has  been  matched  and  accepted  during  its 
curcumnavigation  of  the  ring.  This  accomplishes  the  second 
objective,  cemmunicating  between  processes  regardless  of 
their  physical  location. 

finally,  resource  allocation  is  accomplished  by  a  system 


of  requests  fcr  bids.  If  a  new  process  needs  to  be  set  up, 
a  request  for  bids  describing  the  resources  required  is  sent 
around  the  ring.  Each  processor  having  those  resources  will 
reply  with  a  bid  giving  its  present  resource  availability 
state.  The  reguestox  of  the  bids  will  then  select  the  most 
appropriate  bid  and  confirm  the  new  allocation  with  the 
accepted  bidder,.  Hence  there  is  a  method  of  djnamic 
reallocation  cf  resources. 

This,  then,  is  the  basic  distributed  computing  system 
which  will  be  considered  for  adaptation  to  the  small 
warship* s  needs.  A  more  detailed  description  of  this  design 
can  be  fcund  in  references  1  and  2. 


B.   HABEWABE  CCNSIEEBATIONS 

As  can  be  seen  frcm  the  previous  discussion,  the  heart  of 
the  ring  structured  distributed  computing  system  is  the  ring 
interface.  It  must  have  the  capability  tc  recognize  the 
various  tokens,  maintain  lists  of  processes,  and  convert  the 
sequential  data  format  cf  the  ring  to  the  format  required  by 
its  host.  The  basic  difference  between  the  rings  designed 
at  NPS  [1]  and  at  the  University  of  California  at  Irvine  [6] 
is  the  technology  employed  in  construction  of  the  ring 
interface. 

The  ring  interface  at  Irvine  is  hard  wired.  As  such  it 
enjoys  the  advantage  of  greater  efficiency  of  desigc  and 
hence  higher  data  transmission  rates.  However,  it  loses  in 
flexibility  and  standardization.  It  is  designed  to  interface 
with  one  particular  host  and  to  adapt  such  a  ring  interface 
tc  a  different  host  is  virtually  impossible.  Moreover,  the 
resulting  differences  in  ring  interfaces  designed  for 
different  hosts  can  pose  a  maintenance  problem. 
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The  ring  interface  at  NPS  is  designed  around  a 
microprocessor.  The  ring  speed  in  this  case  is  limited  by 
the  firmware  of  the  programme  stored  in  its  programmable 
read  only  memory  (PROM)-  and  the  access  time  of  this  memory. 
For  the  greatest  flexibility  the  NPS  ring  emplojs  an 
erasable  PBCB.  Thus  if  the  ring  protocol  were  to  be  changed 
or  a  new  test  required  a  different  format  for  exchange  of 
data,  the  microprocessor  memory  could  br  erased  and  a  new 
set  of  firmware  written.  This  entails  a  sacrifice  in  speed, 
hewever,  as  erasable  PSGM's  have  slower  cycle  times  than 
non-erasatle  cnes.  By  using  a  non-erasable  PROM  the  speed 
can  be  increased  but  a  change  would  entail  replacing  the 
memory  with  a  different  chip  containing  the  new  programme. 
In  both  these  cases  the  microprocessor  has  the  advantage  of 
standard  hardware  fpr  the  ring  with  only  a  variation  in  the 
firmware  to  accomodate  the  various  hosts. 

Flexibility  is  of  particular  value  in  the  shipboard 
application  wtere  a  large  number  of  the  hosts  may  be  "dumb", 
such  as  serve  system  controllers  or  monitoring  devices,  or 
may  be  in  existance  at  the  time  of  system  design.  Because 
of  the  advantages  in  maintenance  and  flexibility,  the 
microprocessor  technplogy  will  be  considered  here.  The 
problems  of  ring  data  transmission  rates  will  be  addressed 
again  after  the  requirements  of  the  shipboard  application 
have  been  discussed. 
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II-   ANALYSIS  OF  SYSTEM  REQUIREMENTS 


A.   EXTENT  OP  SYSTEM 


The  first  question  which  must  be  addressed  when 
considering  system  requirements  is  the  extent  of  the  system. 
Hhat  fuDCticns  of  a  warship  should  be  included  for 
processing  in  the  distributed  computing  system?  There  are 
some  areas  trat  are  fairly  obvious,  i.e.  those  that  are 
included  in  current  command  and  control  systems.  These 
include  evaluation  and  processing  of  senscr  infomation 
(radar  tracking,  electronic  warfare  evaluation) ;  tactical 
data  processing  (maintenance  of  track  files,  files  of 
standard  tactical  prpcedures,  etc.);  tactical  communications 
(LINK) ;  fire  control  calculations;  tactical  displays; 
navigation  routines  (ship*s  position,  closest  point  of 
approach  calculations,  course  to  intercept,  etc.) .  But 
there  is  no  reason  to  limit  it  to  these. 

Digital  computer  control  of  main  engines  and  auto-pilots 
are  already  in  existence.  Including  these  in  the 
distributed  computing  system  would  facilitate  automatic 
control  of  emergency  and  evasive  manoeuvres  e.g. 
man-cverfccard ,  torpedo  evasion;  and  also  the  automatic 
inplementaticn  of  tactical  manoeuvres  e.g.  lost  contact 
searches,  zig-zag  courses,  station  keeping.  Eemote 
monitoring  and  automatic  alarm  systems  for  auxiliary 
machinery  exists  as  a  subsystem  of  the  main  machinery 
control.  Expansion  of  this  to  monitoring  other  ship's 
conditions,  smoke  aLarms,  flooding,  etc.   ,   and   making   it 
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part   of   the  distributed  computing  system  would  provide  an 
effective  and  flexible  damage  control  monitoring  system. 


• 


Finally,  communications  is  a  fruitful  area  for 
automation.  Tactical  data  is  already  encoded,  transmitted, 
decoded,  sorted,  displayed,  and  stored  automatically.  The 
problems  of  extension  to  the  remaining  communications  nets 
do  not  seen  overwhelming  and  the  possible  savings  in 
personnel,  paper  and  time  appear  to  be  worth-while.  In 
addition,  the  use  of  a  computer  for  freguency  assignment 
algorithms  would  be  a  distinct  advantage  and  if  this  were 
coupled  *ith  direct  control  of  transmitters  and  receivers, 
electromagnetic  emission  control  would  be  greatly 
facilitated.  Making  this  computer  part  of  the  ship*s 
distributed  computing  system  would  allow  such  probless  as 
the  efficient  handling  of  priority  traffic  to  be  easily 
attacked. 

Such  an  extended  system  is  under  development  ty  the 
Canadian  Ariied  Fcrdes  under  the  title  of  "Shipboard 
Integrated  Processing  and  Display  System"  [7,9].  It  is  the 
adaptation  of  a  ring  structured  distributed  computing  system 
to  the  reguirements  of  this  system  that  will  be  considered. 
Most  of  the  reguirements  in  the  following  sections  are  based 
en  the  preliminary  requirements  for  the  Canadian  system  [9] 
and  are  in  agreement  with  these  requirements. 


B.   DETAIIEE  SUBSYSTEM  REQUIBEMENTS 


1 .   Eisclays 

The  display  reguirements  for  various  departments  on  a  ship 
vary  widely.  Communications  can  be  satisfied  with  purely  an 
alpha-numeric  display.   To   the   engineering  department,   a 
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graphics  capability  would  be  a  desirable  feature.  The 
greatest  demand  undoubtedly  comes  from  the  tactical  displays 
which  are  recuired  tp  present  raw  video  from  various  sensors 
as  well  as  the  synthetic  graphical  and  alpha-numeric  data 
that  are  prcduced  «by  the  various  processors,  i.e.  smcothed 
tracks,  track  designators,  messages,  etc. 

ill  these  requirements  will  affect  the  decisions 
with  respect  tc  display  technology,  i.e.,  what  are  the 
advantages  and  disadvantages  of  raster  scan  versus  vector 
generation  versus  dot  matrix  presentation,  etc.  However, 
with  respect  to  the  system  design  of  a  processing  and 
display  system  ,  most  of  this  is  irrelevant.  What  is 
important  is  the  "intellegence"  of  the  display,  for  it  is 
this  characteristic  that  will  determine  the  type  and 
guantity  of  data  that  must  be  passed  to  a  display  device  to 
maintain  the  presentation  desired  by  the  operator. 

An  allied  aspect  of  display  technology  is  the  method 
cf  maintaining  data  on  the  display  -  whether  a  long 
persistence  cathode  ray  tube  should  be  used  or  the  data 
shculd  be  freguently  refreshed.  The  latter  is  the  only 
practical  sclution  for  any  sort  of  dynamic  display  and  for 
the  rest  cf  the  discussion  it  will  be  assumed  that  the 
display  must  be  refreshed  at  least  30  times  a  second. 
Befresh  rate  then  provides  a  critical  frequency  in  the 
iuplementaticn  cf  the  display  system. 

An  important  dif ferentation  can  now  be  made  in  the 
types  of  data  which  are  to  be  displayed,  i.e.  those  data 
which  may  be  expected  to  remain  constant  for  more  than  one 
thirtieth  of  a  second  and  those  that  may  not.  For  the  latter 
it  makes  no  difference  whether  the  display  is  "intelligent" 
encugh  tc  refresh  itself  or  not.  The  data  will  have  changed 
in  the  interval  between  successive  refresh  cycles. 
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Jcr  the  less  volatile  data  it  is  a  distinct 
advantage  for  the  display  to  be  able  to  store  the  data  for 
the  present  picture.  It  is  then  possible  to  send  to  the 
display  only  changes  to  the  existing  picture.  Otherwise, 
the  data  for  a  complete  picture  must  be  transmitted  to  the 
display  every  refresh  cycle. 

Imposing  this  distinction  on  the  types  of  data  the 
ship*s  system  must  display,  raw  sensor  data  fall  into  the 
volatile  category  while  the  synthetic  data  are  more  stable. 
While  some  displays  are  required  to  present  raw  sensor  data, 
all  have  a  reguirement  for  presenting  synthetic  data.  It 
will  be  assumed  then  that  self-refresh  is  a  desirable 
characteristic  and  will  be  incorporated  into  displays  used 
in  the  distributed  computing  system. 

ibile  the  desirability  of  a  self-refresh  capability 
is  fairly  easy  to  shpw,  the  case  for  further  increases  in 
display  intelligence  is  not  so  clear  cut.  Should  the  display 
have  the  ability  to  .manipulate  data?  At  one  extreme  one  can 
conceive  of  a  tactical  display  that  would  contain  all 
synthetic  data  out  to  the  maximum  range  it  can  accomodate, 
and  all  requests  for  lesser  ranges,  filtered  data,  etc. 
could  be  handled  within  the  display  itself.  This  would 
allow  all  update  messages  to  be  broadcast  to  all  displays 
rather  than  having  to  be  tailored  to  each  display's  current 
presentation  and,  hence,  individually  addressed.  Also,  such 
a  display  would  require  very  little  in  the  way  of 
communication  from  the  display  to  the  system  since  there 
would  be  no  need  fpr  the  system  to  be  aware  of  each 
display*s  particular  presentation.  It  would,  however, 
require  a  large  mempry  and  considerable  computing  power 
within  each  display.  At  the  other  extreme  might  be  a  simple 
routine  so  that  several  sequential  iterations  of  data  (sonar 
range  sweeps,  frequency  scans)  could  be  displayed  such  that 
as  each  iteration  was  displayed  the  remainder   move   up   and 
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th€  oldest  is  discarded.  Intermediate  between  these  two 
eitremes  night  be  the  ability  to  identify  groups  of  data  on 
the  screen  aid  move  the  groups  by  incrementing  the  position. 

It  is  apparent  that  actual  data  rates  are  very 
dependent  on  the  type  of  data  to  be  presented  and  the  amount 
of  processing  that  can  be  done  within  the  display.  The  NATO 
Industrial  Advisory  Group  [8]  is  using  an  average  of  16,000 
bits  per  second  as  the  communication  requirement  for  a 
self-refreshing  display. 

2-   Active  Sensors 

The  active  sensors  employed  in  a  modern  warship  are 
radars  and  active  sonar.  Even  in  a  small  warship  there 
could  be  a  minimum  of  four  to  six  radars;  search,  fire 
control,  and  navigation;  as  well  as  one  or  two  active 
sonars,  hull  mounted  and  variable  depth.  The  raw  data  rate 
of  a  radar  set  can  be  determined  theoretically  by  the 
sanpling  rate  regu^red  to  capture  all  the  information 
available  in  the  bandwidth  of  the  intermediate  frequency 
amplifier  chain.  Since  the  IF  bandwidth  is  in  turn  a 
critical  function  in  the  optimization  of  the  design  of  the 
radar,  the  data  rate  can  ultimately  be  related  back  to  the 
operational  parameters  of  the  radar.  For  a  typical  search 
radar,  pulse  width  cne  microsecond,  the  IF  bandwidth  is  one 
MHz  and  the  data  rate  required  to  ensure  no  less  of  data  is 
thus  1.5  to  2  million  bits  per  second.  A  high  definition 
radar  with  its  shorter  pulse  width  would  obviously  reguire  a 
higher  rate.  Sonar,  due  to  the  more  liesurely  velocity  of 
propogaticn  cf  its  energy,  has  a  much  lower  raw  data  rate  of 
85,000  to  20C,000  bits  per  second. 

It  is  apparent  that  to  multiplex  cne,  let  alone 
several,   active   sensors   on  a  general  usage  data  bus  would 
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impose  a  rather  severe  load  unless  the  bus  speed  were  very 
high.  However,  the  requirement  to  display  raw  sensor  data 
for  evaluation  by  an  operator  must  be  met. 

Heat  to  be  considered  is  the  effect  of  processing 
the  raw  data.  The  f|rst  level  of  processing  would  appear  to 
be  the  autciratic  detection  of  contacts.  This  would  involve 
passing  only  data  en  signals  that  exceed  a  certain  level. 
Here  again  a  typical  situation  will  be  postulated  to  obtain 
ar  idea  cf  reguired  data  rates.  Assume  a  radar  rotating  at 
60  rpm  and  the  possibility  of  200  valid  contacts.  To  obtain 
a  reasonable  probability  of  detection,  the  threshold  level 
for  signals  tc  be  accepted  must  be  set  fairly  low  resulting 
in  a  large  number  of  false  alarms  which  must  be  filtered  out 
by  sweep  tc  sweep  correlation.  Thus,  the  number  of  contacts 
accepted  per  scan  would  be  in  the  order  of  500  to  100C,  On 
each  of  these  a  minimum  of  range  and  bearing,  and  sweep  or 
time  data  must  be  passed.  Even  if  only  10  bits  of  precision 
are  used,  the  resulting  bit  rate  reaches  15,000  to  30,000 
bits  per  secend. 

finally,  the  data  rate  resulting  from  autc-track 
processing  can  be  evaluated.  Here,  only  valid  contact  data 
are  transnitted  and  these  would  include  target 
identification,  x  and  y  coordinates,  time,  course  and  speed. 
Additional  data  need  only  be  sent  when  there  is  a  change. 
Total  data  per  track  may  be  in  the  order  of  200  bits  but  an 
update  messace  would  be  considerably  less  than  that,  and 
would  be  required  much  less  often  than  once  per  track  per 
scan.  Althocgh  the  instantaneous  communication  requirements 
would  be  much  more  variable,  the  average  rate  would  be  much 
lower,  something  in  the  order  of  4000  bits  per  second  for 
the  assumed  criterion  of  200  valid  tracks. 

Ihe  discussion  of  processed  data  so  far  has  been 
limited  tc  radar  data  but  it  should  be  apparent  that  this  is 
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the  limiting  case.  Sonar  contacts  would  te  handled  in  the 
sane  manner  tut  the  expected  number  of  underwater  contacts 
is  in  the  order  pf  one  or  two,  and  ten  cr  more  would  be 
unlikely.  The  communication  load  would  then  te  in  the  crder 
of  tens  cf  tits  per  second. 

Itere  is  one  further  area  of  processing  of  active 
sensor  information  that  needs  to  be  considered,  that  of 
multi-sensor  correlation.  While  autodetection  might  be 
carried  cut  with  a  microcontroller,  auto-track  processing 
has  a  sufficiently  great  computing  requirement  that  a 
ninicomputer  would,  reasonably  be  employed.  Why  not  then 
extend  the  process  to  compare  the  contacts  of  various 
sensors  to  determine  if  the  same  object  had  teen  detected  by 
more  than  ere  senspr?  Hith  appropriate  processing  and 
feedback  to  control  the  sensitivity  of  the  sensors,  greater 
detection  probabilities  can  be  obtained  in  addition  to  the 
immediate  result  of  providing  more  accurate  information  by 
combining  the  data  from  independent  sources.  This  would 
have  the  additional  benefit  of  further  reducing  the  traffic 
on  the  communication  net  since  redundant  information  on  the 
same  contact  tut  originating  from  more  than  one  sensor  would 
be  eliminated.  However,  the  magnitude  of  the  reduction 
would  be  quite  small,  particularly  in  relation  to  those 
obtained  in  going  from  autodetection  to  autc-tracking. 

3-   Ias§iv€  Sensors 

Passive  sensprs  include  acoustic  detectors  as  well 
as  electromagnetic  energy  detectors  for  frequencies  from 
that  of  radar  to  that  of  visible  light.  The  latter  can  be 
broken  into  two  grpups  on  the  basis  cf  the  characteristics 
of  their  out  cut  data.  Electronic  warfare  sensors  cover  the 
freguency  bands  from  roughly  one  to  fifty  GHz.  There  is  a 
considerable  amount  pf  preprocessing  done  and  the   raw   data 
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are  of  little  or  no  interest.  The  second  group,  optical 
systems,  produces  a  linescan  video  image  as  raw  output. 
Included  in  this  group  are  low  light  level  television  and 
infrared  sensors.  As  the  data  from  these  sensors  is 
compatible,  processing  may  include  multi-sensor  correlation 
as  veil  as  digital  enhancement  of  the  image.  The  computing 
requirements  for  this  are  rather  high  and  would  prcbably 
reguire  a  dedicated  minicomputer. 

New  to  consider  each  of  these  three  senscr  groups 
individually.  Passive  acoustic  data  is  normally  presented 
for  human  evaluatipn  after  preprocessing  to  obtain  sound 
intensity  versus  frequency  versus  time  information.  The 
most  common  presentation  method  is  as  an  intensity  modulated 
freguency  sweep  with  the  past  data  being  shifted  along  the 
time  axis  as  each  new  freguency  sweep  is  started.  An 
alternative  coming  Into  greater  use  is  to  censtruct  a  three 
dimensional  graph  with  the  time  axis  appearing  to  recede 
into  the  screen.  The  latter  would  change  less  often,  a  new 
sweep  being  added  every  2  to  5  seconds,  whereas  the 
intensity  modulated  display  would  initiate  a  new  sweep  4  to 
8  times  a  second. 

If  a  systematic  shifting  routine  were  available,  as 
discussed  in  section  II. A. 1,  the  update  would  reguire  in  the 
order  of  5C00  bits  of  information  for  the  intensity 
modulation  and  3=00,000  for  the  graphical  plot.  The 
resulting  data  rates  would  then  be  20,000  to  40,000  bits  per 
second  fcr  the  intensity  modulated  method  (5000  bits  1  to  8 
times  a  second),  or  60,000  to  150,000  bits  per  second  for 
the  graphical  plot  (300,000  tits  every  2  to  5  seconds). 
Having  to  update  the  whole  screen  each  time  would  increase 
the  data  rate  to  several  millicn  bits  per  second. 

The  optical  systems  present  a  volatile  image  that 
will   change   at   a  rate  higher  than  the  30  Hz  refresh  rate. 
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Here  the  criterion  of  IF  bandwidth  can  te  applied  as  in  the 
radar  system  and  data  rates  of  several  million  bits  per 
seccnd  result. 

In  the  cas.e  of  both  passive  sonar  and  the  optical 
systems  much  further  processing  can  be  done,  including 
correlation  with  contacts  held  on  other  sensors  both  active 
and  passive.  The  communications  load  produced  as  a  result 
of  this  is  of  the  nature  of  the  update  and  amplifying 
material  en  valid  contacts,  as  discussed  in  section  II. A. 2. 
Thus  a  data  rate  i<n  the  order  of  10  to  100  bits  per  second 
wculd  be  reasonable. 

E.H.  sensors,  as  mentioned,  prcduce  a  highly 
preprocessed  cutput  uith  the  raw  data  being  of  little  use. 
The  computing  requirements  are  quite  high,  undcuttedly 
requiring  a  dedicated  processor.  However,  the  communication 
reguirement  would  again  be  that  of  passing  update  material 
and  500  bits  per  secpnd  would  be  adequate. 

4 .   I  ear. c n s 

The  neapcns  putfit  of  a  small  warship  would  consist 
of  a  mi*  cf  missiie  launchers  and  guns  and  "soft  weapons" 
such  as  jammers  and  chaff  launchers.  For  remote  control  of 
servo  devices  data  is  usually  supplied  32  times  per  second* 
Even  for  the  most  complicated  launcher  a  few  hundred  bits  of 
data  should  provide  adequate  information  at  each  update  and 
100  bits  would  be  a  reasonable  average.  This  results  in  a 
3200  bits  per  second  average  rate  for  each  weapon  and  even 
kith  8  to  10  weapons  operating  simultaneously  the  total 
average  rate  wculd  be  32,000  bits  per  second. 

It  must  be  considered,  however,  that  these  are  real 
time  systems  and  the  timing  of  the  messages  is  critical.   It 
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will  therefore  be  necessary  to  devise  a  methcd  of  ensuring 
that  weapons  control  messages  do  not  get  delayed  i.e.  left 
waiting  in  ottput  gueues. 

5.   Ccmmunicatiogs 

Communications  has  one  area  in  which  the  needs  are 
already  well  defined.  Computer  controlled  tactical  data 
ccmmunicaticr  via  LItiK  11  has  been  in  use  fcr  several  years 
and  15, COO  tits  per  second  could  be  considered  a  reasonable 
estimate  cf  res  leading  by  this  system  [4]. 

The  reguirements  for  automated  handling  of  general 
message  traffic  are  mere  obscure.  Data  exist  on  the  cumber 
cf  messages  per  day  and  can  be  broken  down  by  priority  and 
classif icaticn.  Storage  reguirements  can  thus  be  easily 
determined,  fcur  million  bits  per  day  should  suffice  and  two 
days*  messaces  should  be  immediately  retrievable  [9]. 
Messages  more  than  two  days  old  can  be  relegated  to  off-line 
storage. 

The  effect  pn  data  bus  loading  is  considerably  less 
clear.  It  can  be  reasonably  assumed,  however,  that  with  the 
exception  of  priority  messages,  general  traffic  will  be  read 
in  periods  of  relative  leisure,  hence,  in  periods  cf  low 
system  activity.  The  effect  of  this  on  bus  loading  should 
then  be  uinimal.  priority  messages  would  go  cut  on  the  bus 
immediately  but  this  type  of  traffic  is  measured  in  messages 
per  hour  if  net  per  day  and  in  terms  of  bits  per  second 
would  seem  to  be  insignificant. 

There  is  one  ether  consideration  and  that  is  the 
case  where  the  direct-access  mass  storage  for  current 
messages  is  a  separate  node  on  the  bus.  In  this  case  all 
message   traffic   would   put   a   load   en   the  bus  as  it  was 


21 


transferred  tc  the  mass  storage  device.  Most  traffic  is 
teletype  at  120  characters  per  minute  cr  16  bits  per  second. 
Even  with  simultaneous  sending  and  receiving  on  several 
frequencies,  the  bus  loading  would  be  in  the  order  cf  100  to 
200  bits  per  second. 

6-   II5£iJl^i2£  Control 

As  mentioned  previously,  the  main  engines  of  some 
ships  are  already  controlled  by  digital  computers,  and  many 
ships  are  steered  ty  autopilots.  The  step  tc  digitizing  the 
engine  and  steering  orders  and  routing  them  via  the  ship 
data  bus  is  a  small  pne  but  opens  up  the  possibility  of  many 
desirable  features.  Instead  of  having  someone  drive  the  ship 
arcund  a  lest  contact  search  pattern  displayed  on  his 
computer  terminal,  the  computer  can  do  it  automatically.  If 
a  tcrpedc  is  detected  en  sonar,  evasive  action  cculd  be 
initiated  automatically  or  by  human  intervention  with 
automatic  execution. 

The   cost  pf  this   would  be  rather  small.   The 

freguency  and  length  cf  propulsion  commands  is  very  Iom,   20 

tc  50   tits  not   mere  often   than   every   2  to  5  seconds. 

Therefore,  peak  loading  would  conceivably  be  50  bits  per 
secend. 

Ihe  machinery  control  computer  would  itself  need  to 
mccitor  status  infprmation  from  the  machinery.  This  would 
invclve  in  the  order  of  100  points  per  second  with  8  to  12 
bits  of  data  apiece  [7].  These  1200  bits  per  second  would 
net  produce  much  of  a  load  on  the  bus,  however,  fcr  ether 
reasons  it  may  not  be  feasible  to  use  the  ship's  bus  for 
this  inf crnaticn.  I-here  is  obviously  a  need  for  a  great 
deal  of  autcnemy  in  the  propulsion  control  system.  As  with 
no  other,  it  is  critical  to  the  survival  of  the  ship.    The 
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advantages  in  making  it  part  of  the  shifts  distributed 
computing  system  are  features  that  are  nice  to  have  but  not 
critical.  lhere  is,  therefore,  a  strong  argument  for 
dedicating  tte  propulsion  system  completely  and  only  using 
the  bus  for  these  non-critical  features,  particularly  since 
it  would  still  permit  the  excess  capacity  in  the  machinery 
control  computers  to  be  used  by  other  systems. 

7-   Ship  Monitor jng 

Under  this  heading  will  be  included  all  the  various 
miscellaneous  devices  throughout  the  ship  that  provide 
information  on  the  state  of  the  ship.  Included  are  existing 
devices  such  as  gyrp-compasses  and  stable  elements,  and 
other  things  which  may  not  be  remotely  monitored  at  present, 
such  as  the  state  cf  auxiliary  machinery,  hull  and  fire 
juiips  and  air  conditioning  units,  and  emergency  warning 
devices,  flccding  indicators  and  smoke  alarms.  There  cculd 
conceivably  te  frcm  several  hundred  to  several  thousand  such 
devices.  Sooe  small  number,  less  than  20  and  including  log 
and  compass,  would  require  several  bits  cf  information  and 
mcritoring  seueral  times  a  second.  The  vast  majority  would 
require  1  tit  and  monitoring  every  few  seconds  to  few 
minutes.  Kith  appropriate  preprocessing  and  data 
ccmpressicn  ty  microcontrollers  before  being  put  on  the  bus, 
the  additional  loading  would  be  in  the  order  of  1,0C0  to 
2,000  bits  per  second  to  monitor  1,000  points. 


C.   S0MMABY 


In  the  preceding  sections  of  this  chapter  the  data  bus 
requirements  cf  the  various  individual  subsystems  have  been 
discussed.  These  data  are  summarized  in  table   1.   The   main 
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TABLE  1 

SUBSYSTEM  COMMUNICATION  REQUIREMENTS 

Subsystem  Bits/Seccnd 

Displays » 16,00G 

Active  Sensors 

Radar  raw... 2,000,000 

processed. U  ,00  C 

Sccar  ra«.« .  150,000 

processed -  2C 

Passive  Sensors 

Scrar  raw., 75,000 

processed.... 5C 

Optical  ra.n 4,000,000 

processed.. 50 

E.S „ 50  C 

Weapons 

Ccrtrol  of  a  single  weapon 3,200* 

Comaunications 

Tactical... 15,00  0 

Geceral <1C 

Propulsicn  Control.......... 50 

Ship  Mcritoring,. . .» 1,000 

♦Real-time  requirement 
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emphasis  has  been  on  interprocessor  conununicaticn  cr  bus 
loading,  since  this  is  the  critical  requirement  in  a 
distributed  computing  system.  The  figures  in  table  1  are 
valid  as  lcng  as,  at  a  minimum,  the  computational 
requirements  of  any  individual  subsystem  can  be  met  in  a 
single  processor.  Several  processes  active  in  one  processor 
might  decrease,  hut  would  in  no  way  increase,  the  bus 
loading. 

The  guestion  of  how  much  computing  power  is  required  for 
the  ship^  sjstem  has  had  little  attention.  It  is  obviously 
an  important  problem  but  it  is  one  which  would  require  a 
much  more  detailed  analysis  of  the  various  function  than  has 
been  attempted  here.  It  is  net  intended  to  discuss  this  in 
depth,  but  a  few  general  comments  can  be  made. 

The  processing  requirements  can  be  met  by  choosing  the 
the  appropriate  number  and  size  of  computers.  While  a 
distributed  computing  system  puts  no  constraint  en  the 
maximum  size  cf  computers  to  be  used  in  it,  cne  of  its  main 
advantages  is  to  be  able  to  use  several  smaller  and, 
therefore,  cheaper  computers,  rather  than  a  large  expensive 
cne.  The  ninimum  size,  as  has  been  mentioned,  is  that  which 
can  handle  the  processing  requirements  for  a  single 
subsystem.  Eaving  exceeded  this  ninimum  size,  the  number  of 
cemputers  en  the  bus  is  transparent  to  the  user  and  the 
software.  therefore,  the  system  designer  is  basicallv  free 
to  vary  the  number  and  size  of  processors  as  he  wishes. 


D.   DETAIIEE  SISTll    REQUIREMENTS 


Having   censidered  the   individual  requirements  cf  the 
various  subsystems  and  components  of  a  distributed  computing 
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TABLE  2 


DATA  DISPLAY  BEQUIREMENTS 


f  anc  tic  i) 

CBT 

Dis| 

:lays 

Hard  Copy 

number 

ran 

synthetic 

Command  and  control 

10 

X 

X 

E.8. 

1 

X 

Sonar,  active 

2 

X 

X 

passive 

1 

X 

X 

Weapons 

3 

X 

X 

Communications 

2 

X 

2 

Propulsion  control 

2 

X 

1 

Ship  monitoring 

2 

X 

TABLE  3 

SYSTEM  BEQOIBEMENTS 

Subsystem  Bits/Second 

Displays   23  X  16000  368,000 

Processed  xadar  4,000 

Processed  sonar  20 

Passive  sonar  50 

Optical  50 

E.i.  500 

Weapons   6  X  3200  19,200* 

Communications,  tactical  15,000 

general  10 

Propulsion  control  50 

Ship  menitpring  1,500 

TOTAL  408,570 
*fieal  time  requirement 
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system,  it  is  necessary  to  consider  the  system  as  a  whole  to 
determine  the  magnitude  of  the  system  (number  of  nodes, 
aggregate  bus  loading,  etc.)  and  any  system  constraints. 
Table  2  summarizes  a  typical  data  display  requirement  for  a 
small  general  purpose  warship.  Since  raw  sensor  data  rates 
are  such  that  a  separate  dedicated  bus  will  be  required, 
those  displays  which  must  handle  this  data  are  identified. 
Table  3  is  a  list  cf  all  systems  tied  to  the  general  purpose 
bus  with  their  estimated  average  communications 
requirements.  The  individual  requirements  are  summed  tc 
give  a  total  average  bus  lead  of  400,000  bits  per  second. 

Note,  however,  that  this  does  not  include  communication 
requirements  associated  with  operating  system  overhead.  This 
requirement  vculd  have  to  be  determined  as  the  system  design 
evolved.  Also  these  are  average  figures,  peak  loading  cculd 
be  much  higher  and  tie  effect  of  real-time  control  messages 
must  be  considered. 

There  is  a  requirement  to  look  at  the  various  subsystems 
from  the  pcint  cf  view  of  autonomy  or  dedication.  Dc  some 
subsystems  because  of  their  importance  or  unique 
requirements  warrant  being  configured  as  autonomous  systems? 
Are  there  groupings  of  subsystems  that  might  have  an 
autonomous  function?  The  importance  of  main  machinery 
control  has  been  mentioned  and  is  considered  sufficiently 
great  tc  justify  autonomy,  probably  requiring  10055 
redundancy  cf  processors  and  complete  cross  coupling.  This 
is  the  only  case  tiat  is  sufficiently  unique  and  important 
for  such  treatment. 

The  arguments  for  grouping  of  subsystems  are  sonewhat 
less  clear.  There  are  several  such  grcups  possible, 
antisubmarine  weapons  and  sonars,  antisurf ace/anti-air 
weapons  anc  radars,  E.W.   and  jammers  and   decoys.    A   case 
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could  b€  presented  for  making  these  groups  autonomous, 
however,  there  is  also  considerable  interchange  of 
information  among  these  systems.  The  technical  aspects  of 
system  integration  aad  reliability  have  a  bearing  on  this 
decision  and  it  Mill  be  considered  in  more  detail  later. 

Finally,  system  reliability  must  te  considered. 
Controlled  degradation  of  perfcrmance  under  failure  is  one 
of  the  greatest  potential  advantages  of  the  ring  structured 
distributed  computing  system.  However,  previous  work  by 
Parber  and  Harris  has  not  had  to  contend  with  one  of  the 
most  obvicus  failure  modes  in  the  military  environment,  that 
of  battle  carnage.  The  ability  of  the  system  to  continue  to 
operate  under  these  conditions  is  of  utmost  importance. 
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III.   SYSTEM  DESIGN 


A.   ABEAS  BECUIBING  CONSIDERATION 

The  requirements  of  a  warship's  computing  system  impose 
several  restraints  cot  alloMed  for  in  the  systems  developed 
ty  Earner  and  Harris-   These  are: 

t.  bus  speed, 

2.  real  tine  processing, 

3.  multiple  addressees,  and 

4.  battle  damage. 

The   effects  of  each  of  these  on  systems  design  will  new  be 
individually  discussed. 

0 

1.   Jus  Speed 

In  a  *arship!s  computing  system  of  the  nature  that 
has  been  discussed,  it  is  critical  that  the  bus  be  able  to 
support  the  data  rate  required  with  little  or  no  delays.  In 
a  system  such  as  designed  by  Farber,  if  the  bus  should 
beccme  overlcaded  for  a  period  of  time  the  result  is  mainly 
inconvenience  to  the  user.  On  a  warship  such  an  overload 
could  be  fatal.  The  bus  data  rate  is,  therefore,  an 
important  factor  in  system  design. 
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The  tus  data  rate  is  determined  by  the  node  design 
and  the  propagation  characteristics  of  the  tus  itself.  It 
is  these  areas  that  Mill  now  be  discussed.  A  node  consists 
of  a  repeater  and  a  ring  interface.  The  bus  consists  cf  the 
repeaters  and  their  interconnecting  wiring. 

Rhile  the  possible  media  for  connecting  the  nodes 
together  are  numerous,  from  a  single  wire  with  ground  leturn 
tc  optical  fibres,  only  three  are  worth  considering  due  to 
problems  cf  electromagnetic  interference  in  the  shipboard 
environment.  Ihese  are  coaxial  cable,  shielded  twisted  wire 
pair,  and  optical  fibres.  All  three  are  capable  of 
transmitting  at  five  to  ten  megabits  per  second  over  the 
cable  lengths  anticipated  on  board  ship.  Optical  fibres  are 
capable  of  ouch  higher  frequencies. 

Ar  independent  problem  that  arises  when  connecting 
several  ccmpnters  together  electrically  is  establishing  a 
common  electrical  ground.  lo  avoid  this  common  ground 
problem  when  using  shielded  electrical  cables,  optical 
isolators  can  be  inserted  into  the  bus  at  each  node.  The 
use  of  optical  fibres,  of  course,  dispenses  with  this 
problem  altogether.  Since  cost,  availability,  and 
convenience  cf  optical  fibres  are  rapidly  approaching  those 
of  coaxial  cable,  they  would  appear  to  be  the  best  choice 
for  the  bus. 

while  the  ring  operating  at  the  University  of 
California  at  Irvine  operates  at  2.25  GHz,  it  uses,  as 
mentioned,  a  hardwired  interface.  The  ring  interface 
proposed  by  Harris  [1]  using  a  microprocessor  was  designed 
to  operate  at  112  kilobits  per  second  bus  data  rate.  This 
was  determined  by  the  memory  cycle  time  (1.1  microsecond)  of 
the  erasable  EBCM  in  the  ring  interface  microprocessor,  a 
requirement  for  the  executicn  of  four  microprocessor 
instructicns  between  bits,  and  a  two  for  one  encoding   ratio 
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of  transnitted  bits  to  data  bits.  The  remainder  cf  the 
circuitry,  being  transistor  transistor  logic  (TTL)  modules, 
is  capable  cf  sustaining  five  to  ten  megabits  per  second 
with  no  difficulty*  Currently  non-erasable  PKCM's  are 
available  with  access  times  of  the  order  of  50  nanoseconds. 
This  would  allcw  the  bus  data  rate  to  be  increased  tc  2.5 
megabits  per  second. 

The  requirements  in  table  3,  for  the  system  being 
considered,  result  in  an  average  bus  data  rate  of  4CQ,00G 
bits  per  seccnd.  The  most  significant  load  (more  than  90% 
of  the  total)  is  that  required  for  displays.  Even  allowing 
for  an  error  in  the  estimate  of  the  average  data  rate  of  a 
factor  cf  three,  a  2.5  MHz  data  bus  still  has  sufficient 
capacity  to  handle  peak  loads  of  two  times  the  average.  If 
this  is  net  capable  of  handling  the  communications 
requirements,  PROMs  utilizing  emitter  coupled  logic  are 
available  with  20  ns  cycle  times  raisinq  the  possible  bus 
rate  to  ever  six  MHz. 

Ihe  ether  side  of  the  interface  must  new  be 
censidered,  the  interface  between  the  ncde  and  the 
processor.  For  purposes  of  discussion  the  characteristics 
of  the  AN/UYK-20  minicomputer  will  be  used,  as  it  is  a 
militarized  minicomputer  in  cemmon  use.  This  computer  uses 
a  16  bit  I/O  wcrd.  Therefore,  at  2.5  million  bits  per  second 
bus  rate  with  16  bit  buffering  in  the  node  the  I/O 
ccntroller  would  reguire  access  to  memory  at  least  once 
every  6.4  Jiicrcsecpnds  or  about  every  eighth  memory  cycle. 
The  computer  can  thus  handle  this  date  rate  continuously 
with  about  a  12. 5%   lpss  of  processing  time. 

If  the  bus  rate  were  raised  to  five  cr  six  MH2  the 
loss  of  processing  time  would  increase  to  25  or  30  per  cent 
for  continuous  sending  and  receiving.  This  still  may  be 
acceptable   if,   in   fact,  a  lower  proportion  of  the  time  is 
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spent  in  communicating.  With  10  to  15  processors  anj  one 
shculd  be  communicating  with  the  bus  only  15  to  25  per  cent 
of  the  tiae  en  the  average.  Thus,  direct  communications 
with  the  ring  interface  from  a  five  MHz  bus  with  only  cne  16 
bit  serial  tc  parallel  conversion  would  still  be  adeguate. 

If  rus  ccmunication  requires  tco  much  processor 
tine,  there  are  several  possible  courses  of  action.  First 
the  AH/OIK-20  is  capable  of  32  bit  parallel  communication. 
This  would  halve  the  time  required  to  service  bus  interrupts 
but  the  buffer  size  in  the  ring  interface  would  have  to  be 
increased.  Second,  a  direct  memory  access  capability  is 
available  in  the  AN/UYK-20  which  would  allow  the  ring 
interface  to  access  memory  by  a  second  port  without  lecking 
out  the  central  processor.  However,  this  would  require 
memory  interface  log4c  in  the  ring  interface.  Finally  a 
first  in  first  out  (FIFO)  buffer  in  the  ring  interface  would 
allcw  more  flexibility  in  the  rate  of  passing  data  between 
the  ring  interface  and  the  host  processor,  i.e.  the 
processor  could  be  interrupted  once  and  a  blcck  of  several 
words  could  be  accepted  or  transmitted  on  consecutive  memory 
cycles.  The  advantage  gained  by  this  technique  would  be 
strongly  dependent  on  the  nature  of  the  processing  being 
carried  cut  as  well  as  the  relationship  between  maximum 
message  length  on  the  bus  and  the  size  of  the  FIFO  buffer. 

Sc  far  no  mention  has  been  made  of  disseminaticg  raw 
data.  Ite  requirement  to  display  this  data  is  present  in 
approximatelj  16-  displays  (see  table  1) .  It  shculd  be 
apparent  frcm  the  discussion  of  the  requirements  that  a 
dedicated  tus  is  required  for  this  information.  Since  a 
single  display  would  only  be  required  tc  display  tte  raw 
output  frcm  cne  sensor  at  a  time,  one  obvious  configuration 
for  this  bus  would  be  a  "star"  pattern.  All  sensors  would 
feed  intc  a  central  switching  unit  which  would  then 
distribute  the  data  to  the  various  displays  as  required.   As 
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this  raw  data  system     is  independent   of  the  distributed 

computing   system   and  would  be  required  no  natter  what  type 

of  computing  system  were  used,   there   will  be  no   further 
discussion  of  this  requirement. 

2-   Seal  Time  Processing 

The  transnission  of  messages  for  real-time  ccntrol 
purposes  poses  another  problem.  If  such  a  message  were  held 
overly  long  in  a  queue  waiting  for  the  ring  interface  to 
gain  ccntrcl  havcc  could  result.  Therefore,  there  must  be  a 
method  whereby  a  real-time  message  can  te  forced  onto  the 
bus. 

Cce  possible  method  could  be  to  have  a  priority 
interrupt  tcken.  A  process  reguiring  to  transmit  a  real-time 
message  cculd  interrupt  the  ring,  transmit  an  interrupt 
token  to  warn  the  rest  of  the  ring  and  the  sender  of  the 
interrupted  message  that  it  had  pre-empted  the  ring,  and 
then  tracsnit  the  priority  message  in  the  normal  manner. 
The  simplest  fellow  up  would  be  for  the  interface 
originating  the  interrupted  message  to  retransmit  it  at  the 
next  opportunity,  the  same  as  if  it  had  detected  a  faulty 
transmissicn  on  an  uninterrupted  message.  A  possible 
problem  arises  fron  this  system,  however,  in  that  ccntrol 
has  jumped  over  the  nodes  between  the  interrupted  sender  and 
the  interrupting  node.  Thus,  instead  of  being  guaranteed  a 
chance  tc  seed  a  message  at  least  once  in  a  time  period 
equal  tc  the  product  ot  the  number  of  nodes  and  the  lengest 
message  time,  a  node  could  be  locked  out.  Tc  preclude  this 
the  interrupting  interface,  rather  than  placing  a  ccntrol 
tcken  on  the  ring  at  the  end  of  its  message,  could  place  a 
special  marker  ontp  the  ring  which  when  recieved  ty  the 
interrupted  node  would  signify  that  it  could  reassert 
ccntrol  and  retransmit  its  message. 
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Another,  but  more  complicated  way  would  have  the 
interrupting  ring  interface  store  the  remainder  cf  the 
interrupted  message  and  retransmit  it  en  completion  of  the 
priority  message.  This  would  have  the  least  detrimental 
effect  on  ring  communications  but  would  reguire  either  a 
FIIO  buffer  in  the  ring  interface  or  full  duplex  operation 
between  the  ring  interface  and  its  host  processor.  A 
variation  en  this  would  be  to  impose  a  fixed  and  fairly 
short  length  en  priority  messages.  A  serial-in  serial-out 
shift  register  would  then  be  used  to  delay  the  interrupted 
message  the  appropriate  number  of  bits.  This  is  perhaps  the 
mGst  pratical  of  the  methods  discussed.  For  example  a  48 
bit  delay  would  allou  three  twelve^bit  control  words  plus  12 
bits  for  interrupt  token  and  addressing.  The  end  cf  the 
priority  message  could  then  be  found  by  counting  bits  rather 
than  needing  another  token. 

3-   Multiple  Addressees 

Cne  cf  the  basic  tenets  of  the  ring  structured 
distributed  computing  system  is  that  interprocessor 
communications  are  addressed  to  processes,  not  processors. 
In  the  ship's  general  purpose  distributed  computing  system 
there  are  a  number  pi  cases  where  the  same  data  is  reguired 
by  several  processes*  for  example:  ship's  ccurse,  rcll  and 
pitch  are  reguired  for  processing  of  sensor  data,  fcr  gun 
control  calculations*  for  manoeuvring  etc. ;  also,  an  update 
fcr  synthetic  video  may  be  used  by  several  displays.  To 
minimize  bus  leading  problems,  it  is  desirable  that  one 
message  fce  sufficient  to  transfer  such  data  to  all  users  of 
it.  There  are  several  possible  ways  of  accomplishing  this. 
One  would  be  to  set  up  a  specific  receiving  process  for  each 
of  these  various  classes  of  data.  However,  this  would  mean 
that  a  process  which  requires  this  information  must  ensure 
that  the  appropriate  receiving  process  were  resident  in   the 
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sane   processor.    Tiiis   would   violate  the  requirement  for 
4-cdependauce  c£  processes  and  physical  locations. 

A  second  ppssibility  would  be  to  structure  the 
process  Dames  to  allpw  qualifiers.  A  general  message  could 
then  be  sent  to  all  processes  with  the  same  qualifier. 
Problems  arise  here  if  the  number  of  qualifiers  is  large, 
then  complicated  decpders  and/or  long  process  names  would  be 
needed. 

Perhaps  the  simplest  solution  would  be  to  allow  a 
prccess  to  have  several  names.  Some  of  these  would  be  the 
same  as  names  of  other  processes  requiring  the  same 
information.  Bost  spftware  would  only  have  to  ensure  that 
all  resident  users  of  the  same  name  were  referred  to  the 
sane  input  buffer  fox  the  common  data. 

** -   Fault  Tole jaoce  Under  Battle  Damage 

A  situation  that  Parber's  distributed  computing 
system  cannct  cope  with  is  that  of  a  failure  in  the  ring 
itself.  Khile  the  failure  of  a  twisted  wire  pair  is  so 
improbable  as  to  be  inconsequential  in  a  non-combat 
environment,  the  probability  of  the  shipbcard  ring  being 
severed  by  battle  damage  is  very  real  and  cannot  be  ignored. 
The  solution  requires  three  separate  actions: 

1.  determination  that  the  ring  has  teen  severed, 

2.  localization  of  the  break, 

3.  repair  or  bypassing  of  the  break. 

The  severing  of  the  ring  will  be  immediately 
apparent   to   the   npde  immediately  beyond  the  break.   Using 
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the  same  encoding  scheme  as  Harris  [ 1  ]  ensures  that  the 
input  cannct  remain  in  the  same  state  fcr  more  than  two 
deck  pulses.  The  only  exception  is  for  a  token,  at  which 
time  the  state  persists  for  three  clock  pulses.  Detection 
of  fcur  cr  acre  identical  bits  in  a  row  wculd  indicate  a 
break.  If  the  detecting  node  then  immediately  started  to 
transmit,  no  ether  npde  would  detect  the  error  and  the  first 
two  steps  cf  the  solution  would  have  been  acconplished. 
what  happens  next  will  depend  on  the  hardware  configuration. 

It  is  obvious  that  some  sort  of  path  redundancy  oust 
be  built  intc  the  ri.ng  to  allow  for  this  situation.  The 
simplest  would  be  to  have  two  or  three  cr  more  ccaplete 
rings.  Per  reasons  to  be  discussed  later  they  should 
connect  the  nodes  in  different  sequences.  All  nodes  wculd 
listen  tc  all  incoming  connections  but  only  transmit  en  one 
output  connection,  all  others  being  held  in  the  same  state. 
If  a  break  were  detected  the  detecting  node  could  seEd  on 
the  broken  ring  a  priority  message  which  would  cause  each 
node  to  switch  to  the  next  back  up  ring.  Simultaneous 
breaks  wculd  be  handled  by  this  system  since  each  node 
imoediatly  downstream  of  a  break  would  initiate  such  a 
message  and  it  would  be  passed  along  tc  all  nodes  until  the 
next  break  was  reached.  The  time-out  system  would  then 
reinitialize  the  riag.  If  the  new  ring  were  not  established 
within  a  specified  lpnger  time  limit  an  automatic  switch  to 
the  next  ling  could  be  incorporated. 
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Figure    1    -      A    SIMPLE    RING 


37 


The  train  weakness  in  this  system  is  the  situation 
where  a  ncde  is  disabled,  This  Mould  interrupt  all  rings 
since  they  all  pass  through  all  nodes.  Furthermore,  if  all 
rings  connected  the  codes  in  the  same  seguence,  the  loss  of 
a  node  is  catastrpphic.  If  the  nodes  are  connected  in 
different  seguences,  ways  can  he  devised  to  use  segments  of 
various  rings  to  bypass  a  defective  node.  The  following 
paragraphs  propose  one  method  of  solving  this  problem. 

figure  1  shows  a  seven  node  ring  with  each  node 
hawing  three  input  aad  three  output  connections.  The  circle 
connects  the  nodes  in  one  seguence  (ring  1) .  The 
connections  shewn  outside  the  circle  (ring  2)  skip  every 
ether  ncde  en  the  circle,  and  the  connections  shown  inside 
(ring  3)  skip  two  nodes.  For  this  arrangement,  as  long  as 
the  number  cf  nodes  is  not  a  multiple  of  two  or  three,  rings 
1,  2,  and  3  will  be  independent  and  complete.  Dummy  ncdes 
can  be  inserted  into  the  ring  to  ensure  that  this  condition 
is  met. 

New  consider  what  happens  if  ring  1  is  in  use  and  a 
break  occurs  between  nodes  three  and  four.  All  interfaces 
are  listening  to  all  inputs  but  transmitting  on  ring  1.  All 
inputs  tc  node  4  are  now  constant  and  it  will  realize  that 
rirg  1  has  been  broken.  One  of  two  things  will  occur  at 
ncde  4  at  this  point,.  If  node  4  has  as  a  host  a  general 
purpose  processor,  it  will  alert  the  processor  and  send  out 
on  the  ring  a  ring  broken  (HBB)  token  and  a  message  saying 
to  stand  by  for  instructions.  If  it  has  a  less  intelligent 
host  (a  gun  ccntroller  or  a  mass  storage  device  perhaps) ,  it 
will  transmit  an  BEE  token  and  its  node  number.  In  this 
case  the  first  node  .having  an  appropriate  hest  will  alert 
its  host  and  ciange  the  EBE  message  to  "wait  for 
instructiens" .  Once  an  EBB  token  has  been  received  a  node 
would   disregard   a   constant  input  until  after  the  ring  had 


38 


been  reestablished.  This  is  necessary  to  avoid  having 
messages  arriving  at  a  node  on  two  different  inputs  at  one 
time. 

The  situatipn  is  now  that  the  first  general  purpose 
processor  downstream  of  the  break  knows  where  the  break  is. 
Moreover,  since  the  prder  of  nodes  on  the  working  ring  would 
be  stored  by  all  processors  each  time  the  ring  is 
established  cr  restructured,  the  processor  would  be  in  a 
position  to  ccntrcl  the  restructuring  of  the  ring. 

Obviously,  the  first  thing  to  try  is  a  switch  to  an 
alternate  rirg.  In  the  example,  node  5  would  take  ccntrol 
and  attempt  to  switch  to  ring  2.  It  would  transmit  this 
command  on  ring  2  and  each  node,  as  it  received  the  ccamand, 
would  retransmit  it  on  the  new  ring.  fihen  the  message 
returned  tc  node  5  it  would  know  reconfiguration  was 
ccuplete  and  reinitialize  the  ring. 

If  ring  2  were  also  broken,  the  message  sent  cut  by 
node  5  wculd  never  return  to  it.  After  an  appropriate  time 
node  5  wculd  transmit  a  command  on  ring  3  to  switch  to  ring 
3  and  wait  for  this  comand  to  return.  If  this  does  not 
occur  then  it  is  apparent  that  all  3  rings  are  broken. 

The  most  probable  cause  of  an  interruption  cf  all 
three  rings  is  that  one  node,  rather  than  three  independent 
ring  segnents,  has  been  destroyed.  In  the  example  the 
lcgical  ccnclusion  is  that  node  3  is  at  fault.  A  ccmmand 
cculd  be  sent  out  by  node  5#  on  ring  1,  and  addressed  to 
ncde  2,  telling  it  alcne  to  switch  to  ring  2.  This  wculd 
bypass  the  presumed  defective  node  3.  Again  the  return  of 
the  message  tc  node  6  would  indicate  that  the  ring  had  teen 
reestablished.  Reinitializing  the  ring  in  this  case  wculd  be 
more  complex,  as  a  node  and  its  processes  have  been  lost. 


39 


Failure  cf  this  last  message  to  return  in  a 
reasonable  tiie  could  initiate  further  attempts  to 
reconfigure  the  rirg.  Perhaps  node  1  could  switch  tc  ring  3 
bypassing  fcoth  nodes  2  and  3.  Eventually  the  processor 
ccntrolling  the  reconfiguration  will  exhaust  all  programmed 
possibilities.  It  would  then  send  out  a  message  identifying 
the  known  servicable  nodes,  in  the  example  only  node  4  and 
itself.  Any  display  downstream  would  receive  this  message 
and  display  it.  Any  processor  downstream  would  add  the 
numbers  cf  the  intervening  nodes.  Thus  the  last  display 
befcre  the  fcreaJc  wpuld  have  as  complete  a  specification  as 
possible  cf  the  contiguous  segment  of  the  ring.  In  the 
example  ring  the  display  at  node  7  would  display  that  nodes 
4  and  5  were  serviceable  and  the  display  at  node  2  would 
display  that  nodes  through  11  and  1  were  serviceable.  Human 
repair  of  the  ring  wpuld  now  be  required. 

It  would  be  possible,  during  the  attempted 
reccnf iguraticn  and  after  reconfiguration  had  failed,  for 
tie  controlling  processor  to  put  control  tokens  on  the  ring 
at  regular  time  intervals  allowing  functioning  processes  in 
the  contiguous  portion  of  the  ring  to  time-share  the  bus  in 
a  degraded  node  in  which  a  node  would  not  expect  to  receive 
its  cwn  message  bade.  Priority  real-time  messages  could  be 
sent  as  usual  and,  of  course,  reconfiguration  control 
messages  wculd  be  sent  as  priorities.  Thus  one-way 
communication  with  processes  downstream  and  before  the  break 
wculd  be  possible.  Whether  this  mode  would  have  any  real 
value  is  debatable  and  would  depend  on  the  physical  location 
of  processes  at  the  time  of  the  break. 

The  actual  number  and  routing  of  interncdal 
connections  wculd  depend  on  the  degree  cf  survivability 
desired,  the  physical  location  of  the  various  nodes  etc.  It 
is  obvicusly  very  dependent  on  the  particular  installation 
and  could  provide  a  major  area  for  operational  analysis   to 
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determine  optimum  values  of  these  parameters.  However,  only 
the  naxinum  possible  number  of  interconnections  per  node 
would  affect  the  software  and  once  this  was  decided  the 
actual  routing  would  not  affect  the  system.  Thus  a  standard 
software  package  cculd  handle  ring  reconfiguration. 


B.   BESOLIING  EESIGN  CONCEPTS 

Appendix  I  certains  a  block  diagram  of  a  ring  interface  and 
repeater  adapted  from  those  proposed  by  Harris  in  fief.  1. 
The  main  chances  are: 

1.  the  addition  to  the  repeater  of  n  continuously 
mcritcred  inputs  and  n  selectable  outputs, 

2.  the  addition  of  a  priority  interrupt  (pri)  and  a 
ring  break  detected  ^BBB)  token, 

3.  the  inclusion  of  an  m  bit  delay  circuit  for 
interrupted  aessage  handling,  and 

4.  the  expansion  of  the  input  and  output  buffers  to  16 
bits. 

These  changes  would  allow  a  ring  structured  distributed 
ccaputing  system  to  operate  in  the  environment  of  a  small 
warship  as  described  in  the  previous  sections.  However, 
there  are  several  otier  areas  that  should  be  considered. 
One  is  the  reguirenent  for  mass  storage. 

On  starting  up  the  ring  or  upon  restructuring  the  ring 
after  battle  damagje,  the  system  must  be  reestablished  and 
available  resources  distributed  as  effectively  as  possible. 
This  reguirenent  plus  the  need  for  any  processor  to  have 
available  a  large  number  of   processes   which   it   might   be 
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required  tc  implement  necessitates  the  existence  cf  copies 
of  these  processes  available  for  loading  by  the  processors. 
These  programmes  m|ght  be  stored  in  mass  storage  devices 
attached  directly  to  the  ring  and  thus  be  accessible  by  all 
the  processors  and  provide  shared  bulk  storage  as  veil. 
However,  for  survivability  there  would  have  to  be  several 
copies  en  several  separate  devices  distributed  arcund  the 
ring.  There  is,  then,  some  justification  fcr  each  processor 
to  have  its  own  dedicated  bulk  storage.  This  would  ensure 
access  tc  bulk  storage  no  matter  what  has  happened  tc  the 
ring  and  would  reduce  bus  traffic  during  reconfiguration. 
When  it  is  censidered  that  the  most  probable  time  that 
reconfiguration  would  be  necessary  is  also  the  time  of 
greatest  system  activity,  i.e.  during  action,  the  latter 
consideration  could  be  very  important. 

Another  area  fcr  consideration  is  the  redundancy  and 
autonomy  built  intp  the  system.  Ideally  it  would  be 
desirable  fcr  all  processors  to  have  access  to  all  inputs. 
Por  example,  all  processors  may  be  capable  of  performing  the 
radar  autc-tracking  function.  However,  this  would  require  a 
dedicated  bus  to  each  of  the  radars.  It  would  be 
impractical  to  provide  this  to  all  processors.  Among  other 
things,  there  are  aot  enough  input  ports  tc  a  processor  to 
handle  all  the  inputs  required.  It  may  be  impractical  in 
seme  cases,  possibly  for  reasons  of  physical  loeatien,  to 
provide  a  particular  input  tc  more  than  one  processor. 
Thus,  there  are  going  to  be  processors  dedicated  to  a 
particular  task.  Ihje  greater  the  number  of  inputs  that  can 
be  made  available  to  more  than  one  processor  the  greater  the 
flexibility  there  can  be  for  reconfiguration  and  hence  the 
greater  tte  survivability  of  the  system. 

The  existence  of  such  dedicated  devices  in  conjucction 
with  the  modes  cf  degraded  operation  have  some  useful 
conseguences.   If,  fpr  example,  a  display  with  access   to  a 
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sensor,  a  processor,  and  the  appropriate  weapon  were 
included  in  that  order  in  the  ring,  as  long  as  a  break  in 
the  ring  did  not  occur  between  them,  the  weapon  could  still 
be  controlled  and  fired.  These  considerations  should  be 
taken  into  account  wiien  designing  bus  routings. 
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IV.   CONCLUSIONS  AND  R2C0MMZNEATI0NS 


The  concept  ox  using  a  ring  structured  distributed 
computing  system  fpr  a  general  shipboard  computing  system 
appears  tc  be  feasible.  While  maintaining  the  advantage  of 
flexibility  in  size  and  configuration,  the  system  can  be 
extended  tc  provide  the  required  bus  speed,  handle  real-time 
processes,  and  ccpe  with  the  additional  hazards  cf  the 
warship  environment.  The  additional  software  overhead  to 
accomplish  this  dees  not  appear  to  be  too  great,  however,  a 
detailed  analysis  would  be  reguired  tc  verify  this 
assumption.  There  is  some  increase  in  the  complexity  cf  the 
node,  but  it  is  still  well  within  the  capability  cf  rhe 
microprocessor  used  by  Harris. 

While  the  system  appears  feasible  there  are  still  aany 
areas  tfcat  require  greater  study.  Neither  the  software  nor 
the  communications  overhead  of  the  system  have  been 
determined.  The  bus  data  rates  used  have  been  averages. 
Thus  a  detailed  lock  at  the  distribution  in  frequency  and 
length  cf  interprccessor  messages  is  required  and  possibly  a 
simulation  needs  tc  be  developed  to  determine  the  capability 
of  the  sjstem  to  handle  peafc  loads. 

Finally,  the  ccsts  and  benefits  of  the  ring  structured 
distributed  computing  system  must  be  compared  with  ether 
forms  of  tie  distributed  computing  system,  particularly  tne 
bidirectional  multiply-connected  data  bus,  to  determine  toe 
overall  relative  value. 
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